As an apparatus using the technique generally called stereo method, there is known, e.g., distance measurement unit (apparatus) for generating distance information from the camera unit up to object to be imaged. In this distance measurement unit, in order to measure three-dimensional coordinate position of the surface of object to be imaged, i.e., three dimensional shape thereof, pictures obtained by imaging, at the same time, the same object to be imaged by plural camera units having different view points are used to determine correspondence points corresponding to each other every respective pixels between those pictures to obtain distance information from the distance measurement unit up to the object to be imaged by that parallax.
In the prior art, there will be explained the representative first technique for generating distance picture consisting of distance information and variable-density picture consisting of luminance information, etc. by camera unit disposed at virtual position and in virtual direction. At the time of generating distance picture and variable-density picture, three-dimensional measurement is first carried out by imaging object to be imaged by means of plural camera units. Then, spatial coordinates (shape) of object to be imaged are determined. Then, an approach is employed to virtually dispose camera unit within that space to generate, by using spatial coordinates (shape) of the object to be imaged, distance picture to be observed by the camera unit virtually disposed, and to generate variable-density picture.
Explanation will now be given in connection with the second technique in which the stereo method is employed to generally determine distance picture with respect to object to be imaged. In this case, explanation will be given in connection with the example where, for the brevity of explanation, reference camera and detection camera for detecting distance between the reference camera and object to be imaged are used to determine correspondence points to generate distance picture.
First, such an approach is employed to image (pick up image of) object to be imaged by reference camera and detection camera disposed at positions different from each other to extract a predetermined area from picture data obtained by the reference camera to successively shift picture data of the extracted predetermined area on picture of the detection camera. At this time, the picture of the predetermined area is shifted on line called epipolar line determined by internal parameters and/or positional relationship of the detection camera. By successively shifting the picture of predetermined area on picture of the detection camera in this way, picture of the predetermined area imaged by the reference camera and picture imaged by the detection camera are compared with each other to determine degree of correspondence. Further, displacement quantity at displacement position where degree of correspondence is the highest is set as parallax at central pixel of the picture of the extracted predetermined area. In addition, by repeating these processing with respect to respective pixels of picture of the reference camera, distance pictures are generated.
Namely, as shown in FIG. 1, object to be imaged is observed by reference camera and detection camera. When it is recognized that point P within the three-dimensional picture is observed at image pick-up point nb by the reference camera, and is observed at image pick-up point nd by the detection camera, it is possible to determine three-dimensional position of the point P. In this case, it is not easy to judge (discriminate) that the corresponding point is image pick-up point nd of the detection camera corresponding to image pick-up point nb of the reference camera. This is called correspondence point problem in the stereoscopic view.
At the time of retrieving (searching) correspondence point generally carried out, image pick-up point nd exists, as is clear with reference to FIG. 1, on line where plane determined by line of sight of the reference camera and angle of visibility of the detection camera and image pick-up plane (surface) of the detection camera intersect (cross) with each other. This line is called epipolar line. Further, when positional relationship between the reference camera and the detection camera and parameters inherent in respective cameras (focal distance or length, etc.) are known, it is possible to determine epipolar line on picture surface of detection camera every respective image pick-up points nb of the reference camera. It is thus sufficient to carry out retrieval of correspondence points on this epipolar line.
An example where correspondence point of image pick-up point nb on picture imaged by the reference camera is detected on picture imaged by the detection camera will now be described. At this time, as shown in FIG. 2, with small area 100 around image pick-up point nb of the reference camera being as template, correlation (correlative) values are determined at several points on the epipolar line of picture of the detection camera. In this case, resolution of the epipolar line, i.e., resolution of distance are six points of image pick-up points nd1 to nd6, and those image pick-up points nd1 to nd6 correspond to distance numbers 1 to 6 corresponding to distances from the reference camera. Further, these distance numbers 1 to 6 correspond to distances on the line of sight from the reference camera. When I(x) is assumed to be luminance value of picture imaged by reference camera and I′(x′) is assumed to be luminance value of picture imaged by detection camera, correlation is calculated by the following formula (1).                               ∑                      i            ∈            W                          ⁢                                  ⁢                                                      I              ⁡                              (                                  x                  +                  i                                )                                      -                                          I                ′                            ⁡                              (                                                      x                    ′                                    +                  i                                )                                                                                  (        1        )            
In accordance with the formula (1), according as correlation becomes larger (higher), degree of correspondence between picture imaged by the reference camera and picture imaged by the detection camera becomes higher, and according as correlation becomes smaller (lower), the degree of correspondence becomes lower. In addition, parameter corresponding to the degree of correspondence is assumed to be evaluation value. The relationship between the evaluation value and respective image pick-up points nd1 to nd6 on the epipolar line is shown in FIG. 3. In this case, according as correlation becomes smaller (lower), the evaluation value becomes larger, and according as correlation becomes larger (higher), the evaluation value becomes smaller.
In accordance with this FIG. 3, image pick-up point nd corresponding to the point in which evaluation value based on the correlation calculated by the formula (I) is the lowest (minimum) is assumed to be correspondence point. In this case, the distance number is “3”. Alternatively, while distance corresponding to the point where evaluation value is the minimum may be determined from respective image pick-up points nd as previously described, there are also instances where interpolation between sampled data is carried out from values at the periphery where the evaluation value is minimum to determine minimum value. By determining distance of picture that the reference camera has imaged from the reference camera and the detection camera in this way, distance picture is generated along with variable-density picture having luminance information.
However, in the first technique for generating distance picture consisting of distance information and variable-density picture consisting of luminance information, etc. by camera unit disposed at the above-described virtual position and in virtual direction, since such an approach is employed to develop (expand) object to be imaged into three-dimensional picture to generate distance pictures by using three-dimensional pictures obtained by carrying out developing (expansion) in a three-dimensional manner, vast amount of calculations are required. Moreover, in this first technique, when three-dimensional measurement is carried out with respect to object to be imaged which has been imaged by plural camera units, it is necessary to image all coordinates which can be observed from the virtual camera unit, and three-dimensional measurements must be thus carried out from many view points. Further, in the first technique, it is also necessary to paste together pictures imaged by respective camera units when three-dimensional picture is generated.
In addition, in the second technique which has been explained with reference to FIGS. 1 to 3, since such an approach is employed to shift picture imaged by the detection camera with picture imaged by the reference camera being as template to retrieve picture to thereby generate distance picture, existence of the reference camera is indispensable.